The present invention relates to an image processing apparatus which receives and recognizes an image of an object, and more particularly to characteristic extraction or segmentation which extracts an object area based on a density distribution characteristic derived from an input image.
In the area of factory automation (FA), there is a strong need to automatize a visual test such as testing of a printed circuit board or testing of foreign material in a medicine by image processing. In response thereto, various image processing (recognition) apparatus have been developed to meet such needs. In such image processing apparatus in which noise is eliminated from an input image during a pre-processing step, an object area is extracted and the object area is recognized. In a test for a defect of a medicine, for example, a tablet area is located from an input image, a defect area of the tablet is extracted and the defect is recognized.
In such an image processing apparatus, much manpower and time are required to develop an object recognition algorithm and program. An example of such a software is disclosed in "Development of General Purpose Image Processing Apparatus-Software" by N. Satoh, T. Gotoh et al., the 28th (first half of 1984) National Conference of The Association of Japan Information Processing, Paper No. 4N-98, pages 995-996. However, a long time is necessarily required to develop the disclosed software even for experts.
With respect to the above processes, the development of a segmentation program to extract the object area from the input image is very difficult hard and time-consuming to a user of the image processing apparatus.
The segmentation means used for the visual test generally comprises three steps: (1) normalization of density-frequency distribution, (2) extraction by feature measurement of local density distribution, and (3) final extraction by geometrical feature measurement.
In the normalization of the density-frequency distribution of the step (1), pre-processing such as elimination of a noise from an input image is carried out. The density distribution of the input image is parallelly shifted to a reference condition based on an average density or a density at a characteristic point such as a peak frequency, or enlarged or reduced to the reference condition based on a density at a characteristic point so that the density-frequency distribution follows an illumination intensity of a test object and a change in a contrast of the test object and a background.
The characteristic point used in the parallel shift may be a minimum/maximum density level of the image, a midpoint of the minimum and maximum density levels, an average density level, a density level for a maximum frequency, a density level for an intermediate one of peaks in the density-frequency distribution, or a density level for a minimum or maximum one of the peaks in the density-frequency distribution, and the characteristic point used in the enlargement/reduction is a combination of a plurality of characteristic points described above. Accordingly, the number of occasions is larger than that for the parallel shift. In searching the peak in the density-frequency distribution, only large peaks may be searched, only small peaks may be searched or both may be searched. The user alternately repeats the development of the program and the experiment to detect the characteristic points to accomplish the normalization of the density-frequency distribution of the step (1).
In the extraction by the feature measurement of the local density distribution, a candidate object area is extracted from a specific density distribution or a two-dimension distribution of the specific density for the density-normalized image of the step (1). Usually, the feature measurement of the density distribution in a partial image is calculated and it is stored as a new density around the partial image. This is called a filtering and space product-sum operation (local density characteristic operation), and it is performed with respect to each pixel of the object image and the results thereof are binarized by a predetermined threshold.
The feature measurement of the density distribution calculated in the partial image may be an average density of the partial image, the number of pixels having a particular density, an average density after horizontal, vertical and diagonal partial differentiation, the number of pixels having a particular density after such a differentiation, the number of pixels having the particular density in the average density image, a maximum, minimum or intermediate density in the partial image or maximum/minimum densities in the partial image. The user alternately repeats the development of the program and the experiment to find an optimum feature measurement in the partial image to accomplish the extraction of the candidate areas based on the characteristic of the density distribution.
In the final extraction by the geometrical feature measurement, only those of the candidate areas extracted in the step (2) which have a specific geometrical feature such as a predetermined range of area, peripheral length or (peripheral length).sup.2 /area are extracted. The step (3) has been well studied in, the past and various practical methods have been proposed. Accordingly, it is not explained here.
The program development works in the steps (1) and (2) take one to two months even if they were done by experts of programming and image processing. If a user having little knowledge on the image processing develops an application program by a trial and error method, a test line of the object product may be substantially changed and the application program may become useless.
The steps (1) and (2) are collectively called segmentation by characteristic of density distribution, or characteristic extraction.
An image processing technique relating to the present invention is an image processing expert system in which an image processing procedure can be determined through repetitive simple interaction with the user by using a prestored knowledge base of image processing experts. Such a system is shown in "Three Types of Knowledge for Image Analysis Expert System" by Tamura and Sakagami, Electronics and Electrical Communication Association of Japan, AL 83-49 (1983), and "Image Processing Expert System" by Sueda, Mikame and Katagiri, TOSHIBA REVIEW, Vol. 40, No. 5, pages 403-406 (1985).
On the other hand, in "Method for Automatically Structuring an Image Processing Procedure by Presentation of Sample Patterns and Application Thereof to Structuring a Linear Pattern Extraction Procedure" by Hasegawa, Kubota and Toriwaki, Electronics and Electrical Communication Association of Japan, PRL 85-38 (1985), a characteristic of an image to be extracted is inputted in a form of image or sample pattern instead of abstract wording. The latter two articles are referenced herein not as prior art but to aid in the comprehension of the present invention.